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■  Benefit 

■  Purpose 

■  Definitions 

■  Thread  Integration  Framework 

■  Thread  Development 
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Benefit 
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■  Standardized  approach 

■  Facilitates  cross-team  collaboration 

■  Consistent  terminology  usage 

■  Improves  experiment  design 

■  Supports  joint  and  service  capability-based  assessment 
framework 

■  Analysis  support 

■  Core  analysts  will  assist  in  application  of  this  approach 

■  Phased  approach 

■  Manageable  workload  for  CDTs 

■  Clearly  identified  milestones  for  thread  development 
products 
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Purpose  of  Operational  Threads 
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■  Facilitate  examining  a  potential  improvement  to  a  deficient 
capability 

■  The  contribution  of  one  or  more  initiatives  or  improved 
infrastructure  either  through 

■  a  new  process  or  modification  to  an  existing  process 

■  a  new  organizationai  construct 

■  a  new  system  or  system-ievei  (i.e.,  “machine-to-machine”)  exchange  between 
databases,  appiications,  or  hardware 

■  Allow  us  to  influence  player  activity  (by  tailoring  scenario 
events)  to  ensure  we  are  able  to  demonstrate  capability  goals 

■  Provide  operational  context  and  therefore  relevance 

■  When  reporting  resuits  (of  initiatives,  capabiiity  goais,  anything 
eise) 

Identify  the  contribution  of  initiatives  to  operationally  significant  activities 

and  processes  (Le.,  operational  threads) 
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Basic  Definitions 
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■  Capability:  The  ability  to  achieve  a  desired  effect  under 
specified  standards  and  conditions  through 
combinations  of  means  and  ways  to  perform  a  set  of 
tasks  (CJCSI  31 70.01  E,  Joint  Capabilities  Integration 
and  Development  System).  Inherent  to  a  capability  are 
the  organizations  and  people,  processes,  and  technical 
means  used  to  accomplish  a  military  task  or  mission. 
Standard  US  Air  Force  capabilities  are  found  in  the 
Master  Capability  Library. 

■  Task:  A  discrete  event  or  action — not  specific  to  a 
single  unit,  weapon  system,  or  individual — that  enables 
a  mission  or  function  to  be  accomplished — by 
individuals  or  organizations  (AF  Doctrine  Center 
glossary.)  Standard  C2  tasks  are  found  in  the  C2  Task 
List  developed  by  the  C2  Capability  Assessment  Team. 
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Basic  Definitions 
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■  Initiative:  Any  potential  solution — across  the  DOTMLPF 
spectrum — for  addressing  a  recognized  warfighting  need  or 
capability  gap.  Initiatives  result  from  the  JEFX  initiative  selection 
process. 

■  Innovation:  Any  deviation  from  an  operational  or  systems 
baseline;  approved  by  Configuration  Control  Board  (CCB) 

■  Capability  Gaps:  The  inability  to  achieve  a  desired  effect  under 
specified  standards  and  conditions  through  combinations  of 
means  and  ways  to  perform  a  set  of  tasks  (CJCSI  31 70.01  E).  For 
JEFX,  this  term  is  synonymous  with  capability  “deficiency”. 
Capability  gaps  are  chosen  by  CDTs  based  on  the  experiment 
focus  areas  and  capability  goal  statements.  CDTs  must  be 
selective  in  choosing  capability  gaps;  time  and  resource 
limitations  often  prevent  us  from  achieving  all  aspects  of  a 
broadly  stated  capability  goal 
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Operational  Thread  Definition 


U.S.  AIR  FORCE 


■  Definition:  A  series  of  operational  tasks  that  relate 
initiatives  and/or  improved  infrastructure  systems  to 
one  or  more  C2  processes 

■  Characteristics 

■  A  design  feature  of  the  experiment;  used  by  Capability 
Development  Teams  to  assess  capability  goals 

■  Should  be  represented  by  an  operational  architecture  and 
supporting  systems  views;  facilitates  transition  within  the 
Joint  Capabilities  Integration  and  Development  System 
(JCIDS)  (OV-6C) 

■  Observable  and  measurable;  defines  specifically  what  the 
assessment  team  will  examine 
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Scenario  &  Vignettes 
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■  Scenario  will  define  level  of  war  (MTW,  SSC,  etc) 
and  Area  of  Responsibility  (AOR) 

■  Also  includes  all  associated  materials  (IPB,  databases, 
background  material,  etc) 

■  You  define  scenario  requirements  for  the  Execution  IPT 

■  Scenario  vignettes  will  be  used  to  sequence 
threads  during  execution 

■  Associated  with  Master  Scenario  Events  List  (MSEL) 
development 

■  Vignettes  are  specific  sequences  of  adversary  activity 
within  the  scenario 

■  Designed  to  elicit  an  expected  response  from  blue  forces 
(i.e.,  operational  thread) 
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Operational  Thread 
Required  Elements 


■  Name  &  Identifier:  Uniquely  identifies  the  thread  (e.g.,  “01A: 
Joint  Air  Estimate  Process”).  Will  also  distinguish  between 
“operational”  and  “technical”  and  among  monitor-assess-plan- 
execute 

■  Description:  Description  of  the  capability  deficiency  or  gap  this 
operational  thread  will  examine.  Describe  any  variations  of  this 
thread  that  will  occur. 

■  Operational  Tasks:  Related  C2  &  ISR  tasks  and  player  activities 

■  Measures:  Characterize  the  performance  of  tasks  and  overall 
effectiveness  of  the  operational  thread 

■  Success:  Capability  gaps;  broad  (e.g.,  shared  awareness) 

■  Effectiveness:  Operational  threads  (e.g.,  time  for  F2T2EA) 

■  Performance:  Tasks;  specific  (e.g.,  accuracy,  timeliness, 
completeness) 

■  Initiatives:  Initiatives  (and  innovations)  that  contribute  to  the 
thread  (i.e.,  potential  “solutions”) 
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Operational  Thread  Integration 


U.S.  AIR  FORCE 


■  A  conceptual  framework  allows  us  to  understand 
the  interrelationship  among  operationai  threads 

■  May  not  incorporate  all  threads  or  all  capability  gaps 

■  May  be  based  on  joint  or  service  doctrine  or  Tactics, 
Techniques  and  Procedures  (TTP) 

■  For  example:  Joint  Air  Tasking  Cycle  or  Joint  Air 
Estimate  Process 
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JEFX  06 
Joint  Air  Tasking  Cycie 
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■  Joint  Air  Tasking  Cycle 
selected  as  operational 
thread  integration 
framework  for  JEFX  06 

■  Based  on  Joint  Doctrine 
(JP  3-30) 

■  Operational  thread 
developed  within  each  of 
the  6  phases  of  this  cycle 

■  Provided  a  recognizable 
structure  for  associating 
threads  to  each  other 


Figure  111-13.  Joint  Air  Tasking  Cycle 
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JEFX  08 

Monitor-Assess-Plan-Execute 


Example  Ops  Thread 

■  Time  Sensitive  Planning 
Process 

■  Measures:  Ability  of  new 
collaborative  tools  to 
improve  SA  in  a 
distributed  planning 
environment 

■  Tasks:  Analyze  Mission, 
Develop  COA,  Develop 
Cmdr’s  Estimate 

■  Sub-Tasks  for  each 
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Example  Operational  Thread 
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Who? 
What? 
When? 
Where? 
With  what? 
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How  do  I  get  started? 


U.S.  AIR  FORCE 


□  Review  capability  gaps 

□  Ensure  shared  understanding  within  your  CDT 

□  Revise/clarify  MOS  as  required 

□  Forms  basis  for  developing  threads 

□  Identify  “broken”  processes,  insufficient  systems  and 
dysfunctional  organizational  structures 

□  Requires  input  from  operational  community 

□  Serve  as  baseline  (“as  is”  situation) 

□  Processes  are  candidates  for  operational  threads 

□  Should  reflect  capability  gaps 

□  Picture  is  worth  1000  words 

□  Determine  best  allocation  of  workload 

□  Requires  knowledge  of  team  member  skills  &  experience 

□  Task  appropriate  sub-groups  to  develop  threads  and  serve  as 
thread  managers 

□  Example:  one  sub-team  per  gap 
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How  do  I  get  started? 
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□  Formulate  propositions  for  the  initiatives 

□  What  gaps,  specifically,  does  each  initiative  contribute  to? 

□  How  do  those  initiatives  contribute?  What  is  important  to 
measure? 

□  What  processes  identified  above  do  the  initiatives  support? 

□  Qualify  your  propositions  as  needed 

□  Break  down  processes  into  steps  (i.e.,  tasks) 

□  Example:  Time  Sensitive  Planning 

□  Add  details  over  time 

□  Eventually  need  measures  for  these  tasks 

□  Where  does  the  process  start  and  end  (for  our  purposes)? 

□  Identify  method  to  cross-check  with  other  CDTs 

□  Will  become  more  important  as  we  proceed 

□  Must  identify  relationship  among  threads  (MAPE) 

□  Will  probably  involve  a  regular  inter-CDT  meeting 
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Additional  considerations 
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■  Planning  or  execution?  (Temporal  focus) 

■  Inter-theater  or  intra -theater?  (Geographic  focus) 

■  Where  do  initiatives  fit  into  this  thread? 

■  How  do  they  support  operational  tasks? 

■  What  measures  are  relevant? 

■  Example:  Network  Centric  Operations  Conceptual  Framework 

■  What  variations  of  this  thread  must  occur? 

■  Under  what  conditions  must  each  trial  occur? 

■  What  scenario  supports  this  thread? 

■  Real-world  or  fictitious? 

■  What  scenario  vignettes  are  required? 
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■  What  level  of  command  does  this  thread  address  (Strategic, 
Operational,  or  Tactical)? 

■  Or  does  this  thread  examine  sharing  of  information  between  levels? 

■  What  phase  of  the  thread  integration  framework  does  this  thread 
address  (Monitor,  Assess,  Plan,  or  Execute)? 

■  Or  does  this  thread  examine  the  transition  from  one  phase  to 
another? 

■  What,  if  any,  live  assets  are  required? 

■  Will  foreign  nationals  participate  in  this  thread? 

■  Or,  is  there  a  portion  of  this  thread  that  is  “US  only”? 

■  Operational  threads  should  be  prioritized 

■  Expectation:  5-8  operational  threads  per  CDT 
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Backups 
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Operational  Thread  Design 

Phased  Approach 


Task 
Development 


Systems  View 
Development 


Initiative 

Selection 


Measure 

Development 


Ops  View 
Development 


Initiative 

Contribution 

Identification 


MSEL& 

Scenario 

Development 


Data  collection 
requirements 
deveiopment 
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Gap  Identification 
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■  Identify  the  specific  capabiiity  gap  or  deficiency  in 
sufficient  detaii  to  ... 

■  Facilitate  call  for  initiatives 

■  Determine  measures  of  success  that  are  associated  with 
each  gap 

■  Focus  of  CDC 

■  Should  be  complete;  reflected  in  the  sub-capability  goai 
statements  and  documentation 

■  Adding  or  revising  capabiiity  gaps  at  this  point  affects  aii 
follow-on  activities 
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Task  Development 
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■  Identify  a  series  of  operational  tasks  (i.e.,  an 
operational  thread)  that  allow  examination  of  each 
capability  gap 

■  Begin  at  a  high  ievei  (e.g.,  1.  Find,  2.  Fix,  3.  Track,  4.  Target, 
5.  Engage,  6.  Assess) 

■  Add  detaiis  and  supporting  activities  overtime,  as  required 

■  Piayer-operator  participation  is  essentiai 

■  Reference  the  Master  Capabilities  Library,  Functional 
Area  Assessment,  AOC  functional  decomposition, 
AFOTTP,  functional  area  CONORS  &  CONEMPS 

■  Most  difficult  step — but  the  most  critical 
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Development 
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■  Identify  measures  for  each  task 

■  Measures  of  performance  for  each  task  and  measures  of 
effectiveness  for  each  operational  thread 

■  Refine  measures  of  success 

■  Developed  at  CDC 

■  Characterize  success  in  achieving  capability  goals 
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Initiative  Contribution 
^  Identification 


■  Identify  the  contribution  each  initiative  wiil  make  to 
the  operational  tasks 

■  Could  lead  to  additional  measures 

■  Initiatives  may  contribute  to  many  operational  threads  and 
associated  tasks 
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Data  Collection 
Requirements  Development 


■  Identify  the  data  that  must  be  collected  during  thread 
execution 

■  Based  on  measures  identified  for  each  operationai  thread 

■  Specified  as  Data  Coiiection  Requirements  (OCRs)  in 
operationai  thread  tooiset 

■  Managed  by  Assessment  iPT 

■  Data  Coiiection  Ceii  (DCC)  is  responsibie  for  coiiecting  aii 
data 
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Capability  Goal 


Related  Assessment  Objectives 


Measures  of  Success 


Initiative 


Initiative  Objective 


Measures  focus  on  operational 
utility  as  well  as  technical 
performance;  selected  on  the 
basis  of  relevance  to  NCO 
attributes 


Operational 
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Initiative  “Core  Capability” 


Measure 


Data  Rqmt 


Who,  What 
When,  Where 
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■  Attribute:  Some  aspect  of  an  event,  situation,  person  or  object 
considered  important  to  understanding  the  subject  under  study 
(DODCCRP  Experimentation  Code  of  Best  Practices).  Exampies  inciude 
range  of  a  weapon  system,  time  required  to  compiete  a  decision  cycie  or 
number  of  nodes  in  a  network. 

■  Measure:  A  standard  by  which  some  attribute  of  interest  (e.g.;  extent, 
dimensions,  or  quantity)  is  recorded.  Exampies  inciude  pounds,  miies  per 
hour,  or  minutes  and  seconds. 

■  Metric:  The  appiication  of  a  measure  to  two  or  more  cases  or  situations. 
Metrics  heip  expiain  variation  in  the  dependent  variabie  across  a  range  of 
vaiues  for  the  independent  variabies. 

■  indicator:  An  indirect  or  “proxy”  measure  when  direct  measurement  is 
not  possibie. 

■  Exampie 

■  Operational  Thread:  Detection  of  target  types  during  TCT 

■  Attribute:  Likelihood  of  detection 

■  Measure:  Percentage  of  detections 

■  Metric:  Relative  probability  of  detection  across  the  interesting  range  of 
conditions  (target  type,  sensor  array,  weather) 
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Developing  Measures 
Levels  of  Measurement 
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■  Nominal  Measurement:  Assignment  of  observations  to  categories, 
when  the  categories  have  no  natural  order.  Examples  include 
gender  and  nationality. 

■  Ordinal  Measurement:  Categories  have  a  natural  order,  but  the 
distance  between  them  has  no  meaning.  Examples  include  threat 
levels  (low,  medium,  high)  or  level  of  training  (novice,  journeyman, 
expert). 

■  Interval  Measurement:  Distances  between  points  on  a  scale  are 
meaningful,  but  they  are  anchored  arbitrarily  (i.e.,  zero  has  no 
empiral  meaning).  Example  includes  temperature  or  IQ. 

■  Ratio  Measurement:  Equal  level  intervals  and  a  meaningful  anchor 
point.  This  is  the  preferred  level  of  measurement,  if  possible. 
Examples  include  time  to  complete  a  task,  weapons  range,  or 
years  of  experience. 

Avoid  falling  into  the  trap  of  over-precision 
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